Accelerating, hyper-accelerating, and decelerating probabilistic networks 
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Many growing networks possess accelerating statistics where the number of links added with each 
new node is an increasing function of network size so the total number of links increases faster 
than linearly with network size. In particular, biological networks can display a quadratic growth 
in regulator number with genome size even while remaining sparsely connected. These features 
are mutually incompatible in standard treatments of network theory which typically require that 
every new network node possesses at least one connection. To model sparsely connected networks, 
we generalize existing approaches and add each new node with a probabilistic number of links to 
generate either accelerating, hyper-accelerating, or even decelerating network statistics in different 
regimes. Under preferential attachment for example, slowly accelerating networks display stationary 
scale-free statistics relatively independent of network size while more rapidly accelerating networks 
display a transition from scale-free to exponential statistics with network growth. Such transitions 
explain, for instance, the evolutionary record of single-celled organisms which display strict size and 
complexity limits. 
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I. INTRODUCTION 

The rapidly expanding field of network analysis, re- 
viewed in 

0111a, 

has provided examples of networks 
exhibiting "accelerating" network growth, where total 
link number grows faster than linearly with network size 
P|. For instance, the Internet and World Wide Web ap- 
ear to grow by adding links more quickly than sites 
|(| though the relative change over time is small 0. 
Similarly, both the number of links per substrate in the 
metabolic networks of organisms Q and the average 
number of links per scientist in collaboration networks 
increases linearly over time H IH IH IH 13 , while also 
languages appear to evolve via accelerated growth [l4| . 
These studies have motivated examinations of stochas- 
tic accelerating networks where the number of new links 
added is probabilistic (though integral and greater than 
one) causing network transitions from scale-free to expo- 
nential statistics [13, EU Us UJ1 C3 • 

These previous studies of deterministic or stochastic 
networks have typically considered networks where every 
node has at least one connection which constrains the 
rate of acceleration that can be considered. In partic- 
ular, if each new node added to a network of N nodes 
is accompanied by N a > 1 new links with acceleration 
parameter a > 0, then ensuring that the network is less 
than fully connected constrains acceleration parameters 
to the range < a < 1 [6( . Equivalent limits were consid- 
ered in Ref. ^ij- However, real networks can contain a 
majority of nodes that are entirely unconnected — many 
computers are unconnected or only intermittently con- 
nected to the Internet and many people are unconnected 
nodes in social or sexual contacts networks. Modelling 
networks where a substantial proportion of the nodes are 
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unconnected requires a probabilistic approach in which 
the number of links added with each new node scales as 
pN a for some probability parameter < p < 1. The in- 
troduction of a probabilistic envelope lifts constraints on 
the acceleration parameter allowing networks displaying 
deceleration a < 0, no acceleration a — 0, acceleration 

< a < 1, and hyper-acceleration a > 1. 

Hyper-accelerating probabilistic networks with p <C 

1 and a > 1 are not merely of theoretical interest, 
and appear for instance in prokaryote gene regulatory 
networks where many gene nodes are unregulated and 
merely constitutively or stochastically expressed [2(], • 
Prokaryote gene regulatory networks are sparsely con- 
nected (p <C 1) and display hyper-acceleration a = 1 as 
established by independent comparative genomics analy- 
ses [HElElEiliGl. This is likel y due t0 their reliance 
on sequence homology interactions between protein tran- 
scription factors and specific promoter binding sequences 
sites [26|. In these regulatory networks, outbound regula- 
tory links are preferentially attached to existing regulator 
nodes as gene duplication events contribute to up to 75% 
of each of new genes [22J l28ll29Ll30l l3ll 132 . l33| and regu- 
latory transcription factors |3J,|35j. In contrast, inbound 
links to regulated nodes are randomly formed due to the 
random drift of gene promotor sequences, although their 
subsequent fixation is determined by selection. The high 
acceleration parameter a — 1 ensures that these prokary- 
ote gene regulatory are size and complexity constrained 
at predicted limits which closely match geno mic size lim- 
its observed in the evolutionary record |20l l2ll |2(j . Ac- 
celerating networks are more prevalent and important in 
society and in biology than is commonly realized. In fact, 
any network in which the dynamical state of any node 
depends on the immediately preceding state of (ideally) 
every other node is accelerating and will perhaps display 
either structural transitions from randomly connected, 
to scale free statistics, to densely connected and perhaps 
finally to fully connected statistics, or explicit size and 
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complexity constraints 36] . 

This paper generalizes previous analysis to con- 
sider probabilistic accelerating networks in decelerating 
regimes with a < 0, accelerating regimes with < a < 1, 
and hyper- accelerating regimes where a > 1. We will 
show that hyper-accelerating networks under preferen- 
tial attachment of new connections to existing nodes can 
display transitions in their connectivity statistics from 
scale-free to exponential statistics dependent on the ac- 
celeration parameter a, the probability parameter p, and 
network size N. These transitions occur at the point 
where the accelerating growth in connection number can 
no longer be sustained introducing exponential cut-offs 
in the connectivity distributions. These cutoffs can be 
generated by different mechanisms and appear in grow- 
ing networks subject to strong aging effects , in finite 
sized networks |38| . and in information filtered networks 

mm 

To complete our examination of network transitions in 
accelerating, hyper-accelerating and decelerating prob- 
abilistic networks, we first define our growing network 
models using undirected links in Section [H] This defi- 
nition allows an immediate rough quantification of the 
probability of an accelerating network forming a single 
giant connected component or of undergoing a transi- 
tion to a fully connected regime with network growth. 
In Section Hill we examine probabilistic networks grow- 
ing through the preferential attachment of new links to 
existing connected network nodes, while in Section IIVI 
we examine growing probabilistic networks featuring the 
random attachment of new links to all existing nodes. 



II. PROBABILISTIC ACCELERATING 
NETWORKS 

We consider networks growing through the sequential 
addition of numbered nodes rik for 1 < k < N where at 
network size fc, node rii (1 < i < k) has In- undirected 
links. The addition of node and its Ikk links will in- 
crease the probable number of links attached to existing 
nodes rii for I < i < (fc — 1) so Uk > la. We assume new 
links are added only with each arriving node (so no new 
links are added between established nodes), and that the 
average number of new links attached to node on its 
entry to the network is a function of network size 



f t?- (1 - Nl+a ) ~ a<-l 



hk = pk c 



(1) 



where the probability constant p satisfies < p < 1. This 
average is then a decreasing function of network size for 
a < 0, constant for a — 0, and an increasing function of 
network size for a > 0. The total expected number of 
links L in a probabilistic network of large size N is then 
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a > -1. 



Consequently, the expected number of connections is C = 
2L, and the average connectivity per node is (k) = C/N. 
In later numerical simulations, we compare the statistics 
of networks of different sizes which is possible provided 
they possess the same average connectivity. We achieve 
this for accelerating networks with a > by choosing 
different probability parameters p for each different sized 
network according to 
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This allows us to later consider different networks with 
identical average connectivity (k) — 0.4, where networks 
have either a critical size of iV c = 1000 nodes connected 
by L = 200 links with p = 0.2(1 + a)10~ 3a , or a critical 
size of N c = 10, 000 nodes connected by L = 2, 000 links 
with p = 0.2(1 + a)10" 4Q . 

Because link formation is probabilistic, the total num- 
ber of links attached to node nk on entry to the network, 
denoted j say, ranges between and k, so < j < k with 
average pk a . As each link either forms or does not form, 
we model the link formation process using a binomial 
distribution. For the binomial distribution, the average 
hk = pk a equates to the product of the maximum num- 
ber of links k and the link formation probability. Con- 
versely, the size dependent link formation probability at 
network size k equates to lkk/k = pk a ~ l . Consequently, 
the probability distribution that node rik forms exactly 
j links on entry to the network is 



P(j,k) 



(pk a - x ) j [1 -pk a ~ x ] 



k-j 



(4) 



That is, when a = the latest node nk can poten- 
tially form links with every one of the k existing nodes 
(counting itself) with a network size dependent proba- 
bility p/k to give an average number of new links of p 
connections with each new node. When a = 1, the lat- 
est node rife can potentially form links with every other 
node with constant probability p to give an average num- 
ber of new links of pk with each new node, and so on. 
The connection probability is constrained to be less than 
unity, pk a ~ x < 1, or equivalently, the average number 
of links added to node is required to be less than k, 
pk a < k. For constant p, hyper-accelerating networks 
with a > 1 will eventually violate these constraints with 
network growth which effectively imposes size and struc- 
tural constraints on these networks. 

There are two types of network connectivity transi- 
tions which have been of interest in the literature. The 
first occurs when a growing network changes from sparse 
to dense connectivity as this change necessarily implies 
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that the final connectivity statistics will become exponen- 
tial in nature. That is, while a sparsely connected net- 
work can possess either scale-free or exponential statis- 
tics, a densely connected network must exhibit exponen- 
tial statistics (as multiple connections are not allowed 
between any two given nodes). This transition occurs for 
undirected links when approximately L > N 2 /2. The 
second transition of interest occurs when the connected 
nodes coalesce to form a single giant island of intercon- 
nected nodes. To borrow results from random graph 
theory |4l| (which have been roughly validated for ac- 
celerating biological networks pel l2l|1. the giant island 
is expected to form when approximately L > N/2. We 
now investigate the dependence of these transitions on 
the acceleration parameter a. 

It is evident that decelerating networks with a < are 
very sparsely connected so neither of the dense connec- 
tivity or giant island transitions will occur. Hereinafter, 
we only consider accelerating networks with a > 0. For 
such networks, the dense connectivity and giant island 
transitions will occur respectively at the points 



L > 
L > 



AT2 

2 
N 
~2 



N a ~ 1 > 



N a > 



1 + a _ 
1 + a _ 



(5) 



This establishes that, for instance, non-accelerating net- 
works with a — can never undergo a transition to a 
densely connected regime for any choice of the probabil- 
ity parameter p for large network sizes N, though these 
networks can form giant islands with network growth as 
long as p > 1/2. This inability to become densely con- 
nected remains for all accelerating regimes a < 1, but 
dense connectivity can emerge in the hyper-acceleration 
regime. At this point with a = 1, growing networks 
can undergo a transition to the densely connected regime 
when the probability parameter is large, p ~ 1. For yet 
higher acceleration rates a > 1, all growing networks un- 
dergo a transition to the dense connectivity regime and 
display single giant islands, and these transitions occur 
at network sizes determined by the probability parameter 
P- 

In the following sections, we determine the connectiv- 
ity statistics for accelerating and hyper-accelerating net- 
works under both preferential and random attachment of 
new links to established nodes. 



III. PREFERENTIALLY ATTACHED 
ACCELERATING NETWORKS 

We now turn to consider accelerating and hyper- 
accelerating networks (a > 0) where new links are princi- 
pally preferentially attached to existing connected nodes. 
The final generated link distribution will depend on the 
balance of different countervailing trends, namely the re- 
spective weights given to preferential versus random at- 



tachment, and the magnitude of the acceleration param- 
eter which influences both the rate at which links are at- 
tached to newer nodes and the rate at which while older 
nodes accumulate links. We now examine these trends in 
detail. 

On entry into the network, node rik establishes an 
average of Ikk = pk a links with existing nodes rij £ 
{ni, . . . ,rifc}. We suppose that these links either are pref- 
erentially attached to node rij with probability propor- 
tional to rij's current connectivity Ljk, or are randomly 
attached to rij with probability proportional to j3, a ran- 
dom connection parameter. Using the continuous ap- 
proximation pH 1421 l43j ] . the rate of growth in link number 
for node rij is 



dljk 
dk 



= I 



A- A- 



I { l jk 



(6) 



jo v-jk <-/3) dj 

Here, the rate of growth in the connectivity ljk is de- 
termined by the number of new links added with node 
n% (i.e. Ikk) which can be either preferentially attached 
to the existing nodes rij with probability proportional to 
that nodes current connectivity (i.e. ljk), or randomly at- 
tached if {3 > allowing even initially unconnected nodes 
to establish connections. (The case (3 = ensures that 
a node that receives zero links on entry to the network 
remains unconnected for all time.) 

The denominator of Eq. El is a probability weighting 
to ensure normalization and is roughly equal to the total 
number of connections (C) for all nodes at network size 
k. Following we can evaluate the denominator using 
the identity 



ml ljkdj 



01 



dj + l k . 



(7) 



y dk 

which can be evaluated using Eq. to give 
d f k 

J ljk dj = 2l kk - (8) 

Noting Ikk ~ pk a , this can be integrated to determine 
the denominator of Eq. [S]to be 



(ljk + (3) dj 



2p 



1 



k 



a+l 



(3k. 



(9) 



The first term on the RHS is the total number of connec- 
tions at network size k and is in agreement with Eq. [2] 
Substituting this relation back into Eq. gives 

dljk _ pk a -\l 3k +l3) 
dk 



(10) 



The connectivity statistics can readily be obtained by 
integrating Eq. 1101 with the initial conditions Ijj w pj a 
at time j to obtain 



(P + P)(f)' 



a 



0. 



(11) 



(pj a + P) 



(3, a > 0. 
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Differentiation of these curves with respect to network 
size N satisfies Eq. [5] while integrating these link num- 
bers over all node numbers j (numerically for a > 0) 
gives the required total number of links as in Eq. [21 
Further, differentiating the linkage distribution Ij^ with 
respect to the age index j establishes that average node 
connectivity is monotonically decreasing with increasing 
node age j for < a < 1, flat for a = 1, and monoton- 
ically increasing for a > 1. In other words, irrespective 
of the choice of the (3 parameter, slowly accelerating net- 
works with < a < 1 have the majority of their con- 
nections in the oldest portions of their networks, while 
hyper-accelerating networks with a > 1 place the major- 
ity of the connections in the youngest portions of their 
networks. The transition point occurs at a = 1 when 
connectivity is uniformly distributed over the network 
irrespective of node age. 

The further analysis of this age dependent connectiv- 
ity distribution is heavily dependent on the setting of 
the random choice parameter (3. Self-evidently, setting 
(3 small ensures that preferential attachment processes 
dominate while the choice of large (3 ensures that ran- 
dom attachment dominates. Appendix ^ uses a Taylor 
series expansion to demonstrate that preferential attach- 
ment dominates hyper-accelerating networks only when 
(3 is so small as to be essentially zero, and hereinafter, we 
analyze preferential attachment using the setting (3 = 0. 

For convenience, we note that the choice (3 = sets 
the connectivity versus age distribution to be 



l jN = P N 2 j- 



(12) 



for arbitrary a. The choice a = duplicates existing 
results found for growing networks which add a constant 
number of links with each new node subject to prefer- 
ential attachment |2| , while additionally, choosing p = 1 
duplicates the deterministic results of Ref. m the 

regimes < a < 1 considered in that paper. This purely 
preferentially attached distribution is shown in Fig. ^ 
along with example simulation runs at different accelera- 
tion parameters a. (The floating free end of the first link 
to the first connected node is randomly attached to any 
of the existing nodes as they all possess zero links reduc- 
ing preferential attachment to random attachment.) Part 
(a) of this figure shows the average connectivity ljN of 
node rij which is monotonically decreasing with respect 
to node age for < a < 1 and monotonically increasing 
when a > 1. These different trends depend on whether 
the accumulative effects of preferential attachment out- 
weighs the accelerating number of added links or not. 
The setting a = 1 ensures the average number of links 
per node is independent of node number as preferential 
attachment exactly cancels the bias in initial link num- 
ber towards later nodes. Part (b) of this figure shows 
the actual connectivity of node rij (rather than the aver- 
age connectivity) in example simulated networks, mak- 
ing it evident that actual node connectivity is generally 
a monotonically decreasing function of node age. Why 
this is so is discussed later. 
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FIG. 1: Snapshots of the non-stationary statistics of pref- 
erentially attached growing networks all possessing an aver- 
age number of links per node (k) = 0.4 when at network size 
N — 1000. (a) The average connectivity distribution ljN as 
a function of node age j for various values of the accelera- 
tion parameter a. Connectivity is monotonically decreasing 
for < a < 1, fiat for a = 1, and monotonically increasing 
for a > 1. (b) Example simulated networks for various ac- 
celeration parameters a with nodes listed sequentially left to 
right from n\ to un with the connectivity of each node indi- 
cated by vertical lines. The horizontal dashed line indicates 
an average connectivity of two. 



The ljN distribution contains information about both 
node connectivity and node age and so approximates net- 
work statistics (simulated or observed) when all of this 
information is available. However, it is usually the case 
that node age information is unavailable necessitating 
calculation of connectivity distributions that are not con- 
ditioned on node age. This effectively requires binning 
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together all nodes irrespective of their age to obtain a 
final link distribution. Following Ref. [2(|, the usual 
continuum approach 0, 0. D3 must be modified when 
non-monotonically decreasing connectivity statistics are 
present. In particular, the final connectivity distribution 
is obtained via 



C(k,N) 



1 

N 



= ^ 



N 



dj S(k - l jN ) 



01 



dj 



a t\j=j(k ) N)), (13) 



where j(k,N) is the solution of the equation k = Ijjy. 
The top line is used when all nodes possess the same 
average link number while the second line is applicable 
with the plus (negative) sign when the average numbers 
of links per node is monotonically increasing (decreasing) 
with node number. Non-monotonic cases require alter- 
nate approaches. 

We now use Eq. ^]to calculate the age independent 
final link distribution C(k, N) relevant when age informa- 
tion is unavailable. In the case a = 1 we have j ^ j(k), 
meaning the connectivity k is independent of the node 
age j, so the delta-function of Eq. 1131 immediately inte- 
grates to give L dj 5(k—pN)/N — 6(k—pN). For other 
cases, we have the constraint 



J 



2 l + o 



(14) 



The age constraint 1 < j < N translates into the connec- 
tivity constraints k G \pN a ,p/V( a+1 )/ 2 ] for < a < 1, 
and k G [pN ia+1 ^ 2 ,pN a ] for a > 1. Via Eq. [U the fi- 
nal continuous connection distribution under preferential 
attachment is then 



C(k,N) = \ S{k-pN), 



3-c 
fcT^ 



a-1 _ a_ _ J 



< a < 1, 
a = 1, 
a > 1. 



(15) 



Each of these separate distributions is normalized over 
k G [fc ,fcoo] via C(k,N) = 1 where k G [pN a , oo) 
for < a < 1, k G [0,7V] for a = 1, and A: G 
[0,p./V a ] for a > 1. The average connectivity per node 

(k) = kC(k, N) = C/N as required over these same 
ranges. Again, the choice p — 1 and < a < 1 duplicates 
the deterministic results of Ref. [l!j . 

The resulting age-independent connectivity distribu- 
tions C(k,N) are shown in Fig. [21 and are monotoni- 
cally decreasing for acceleration parameters < a < 1, 
monotonically increasing for 1 < a < 3, uniformly flat 
for a — 3, and monotonically decreasing for a > 3. In 
all cases except a = 1, the age independent distribu- 
tion C(fc, N) is a power law proportional to fc -7 with 
7 = [a — 3)/(a — 1). The exponent here can take the 
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FIG. 2: T/ie age-independent connectivity distribution 
C(k,N) as a function of acceleration parameter a. The log 
plot means that straight lines indicate power-law distributions. 
The connectivity distributions are monotonically decreasing 
for < a < 1 over the range k £ \pN a , oo), a delta function 
for a — 1, and over the range k £ [0,pN a ], are monotonically 
increasing for 1 < a < 3, flat for a = 3 and monotonically 
decreasing for a > 3. Networks in this figure possess an av- 
erage connectivity per node of (k) = 0.4 at a network size of 
iV = 10,000. 



values 7 G [3, oo) for a G [0,1) and 7 G (— 00, 1) for 
a G (l,oo), so no value of the acceleration parameter 
permits exponents in the range 7 G [1,3]. 

The continuous connectivity distributions C(k,N) are 
typically defined over very limited ranges k G [k , fcoo] so 
it can often be the case that every node of the network is 
predicted to have fractional connectivity less than unity. 
Of course, it is impossible to observe nodes with frac- 
tional connectivities — every node either has zero connec- 
tions or one connection, or two connections, and so on. 
When most connected nodes have fractional connectiv- 
ity, it is necessary to convert the unobservable contin- 
uous distribution C(k,N) to an observable discrete dis- 
tribution, which we denote Ck, giving the proportion of 
network nodes possessing an integral number of k links 
for k G [0,7V]. For networks with connectivity distribu- 
tions which are long tailed, the continuous distribution 
C(k, N) closely approximates the discrete distribution Ck 
and this step is often not required. In Appendix [51 we 
detail how to calculate the observed integral connectivity 
distribution Ck generated from a continuous power-law 
connectivity distribution C(fc, TV) defined over a finite 
range k G [fco^oo]- It is generally the case that finite 
ranges impose exponential cutoffs on scale-free continu- 
ous distributions C(k, N) to give discrete distribution Ck 
possessing exponential statistics. A number of alterna- 
tive mechanisms can impose an exponential cutoff on a 
power-law connectivity distribution |3?1 l38l I39I ] , and we 
now add hyper-acceleration to this list. 

Appendix iBl specifies a numerical algorithm for calcu- 
lating the observable discrete distribution Ck defined by 
the unobservable continuous distribution C(k,N). The 
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last stage of the calculation, consider the case a = 1 
where we have kj — PjN — pN and pj = p for all nodes 
< j < N — 1 w N, and noting the absence of any 
contribution from the right of b so C + = 0, we have 



Ck — C, 



p k (l 



p) 



N-k 



1, 



(18) 



1.2 1.4 



FIG. 3: The age-independent discrete connectivity distribution 
Ck as a function of acceleration parameter a under preferen- 
tial attachment. It is evident that while the unobservable con- 
tinuous distributions C'(k, N) appear to be distinguishable, the 
observable discrete distributions are virtually indistinguishable 
at low connectivity numbers. 



calculated discrete distributions equivalent to the contin- 
uous distributions of Fig. [2]appcar in Fig. [3] demonstrat- 
ing that the generated discrete distributions are virtually 
indistinguishable over wide ranges of the acceleration pa- 
rameter in marked contrast to the very different shapes of 
the continuous distributions C(k, N) at different acceler- 
ation parameters. This is due to the exponential cutoffs 
imposed when the continuous distributions are defined 
only over finite ranges. All of the calculations involved 
in the numerical algorithm of Appendix [H] can in prin- 
ciple be done by hand as we now illustrate using distri- 
butions C(k,N) where the boundary point b lies within 
the range of support ko < b < koo for low acceleration 
regimes < a < 1, but outside the range of support 
ko < koo < b for high acceleration regimes a > 1. Using 
Appendix El this choice gives 



for k G [0, N}. 

As mentioned above, Fig. ^ makes it apparent that 
the actual connectivity of connected nodes generally de- 
creases with node age even though average node connec- 
tivity can be monotonically increasing or decreasing. The 
reason is that the IjN curves average connectivity over 
all nodes rather than only over connected nodes. (Recall 
that j3 = means that nodes that are unconnected on 
entry to the network forever remain unconnected.) The 
probability that node rij is connected is straightforwardly 
p cti) = 1 - P(0,j) using Eq. E|or 



Pcti) - 1 - (1 



PJ 



PJ 



(19) 



Hence, the average link number per node at node rij (Eq. 
I12[l equates to the product of the average number of links 
per connected node at node rij, denoted l c (j,N), and 
the probability that node rij 
definition then, we have 



is connected, or P c (j)- By 



giving 



lc(j,N) 



l jN = lc(j,N)P c (j), 



1-(1-PJ 



(20) 



(21) 





l-a 




1- 


fc-1/2 




fc+1/2 





ct = ( 



Consequently, the average number of connections of con- 
nected nodes l c (J,N) is always a monotonically decreas- 
ing function of node age j. (An equivalent derivation 
appeared in Ref. [2fT|.) 

It is difficult to further analyze the connectivity distri- 
bution for actually connected nodes as the distribution 
< a < 1, k G [2, co), f Eq |^is only incompletely sampled — node rij is con- 

a=l,ke [0, AT], 



0, 



a > 1. 



(16) 

All of these distributions then have nonzero support to 
the left of the boundary b and so can be partitioned to 
give NC~ nodes labelled < j < (NC~ - 1) each with 
average connectivity kj — pjN equal to 



pN, 



< a < 1 



a = 1, 



a > 1. 



(17) 



These values then feed directly into Eq. IB 111 to give the 
total discrete distribution Ck- To further illustrate this 



nected only with probability 1 — P(0, j). Fig. 01 shows 
age independent connectivity statistics for simulated net- 
works at a snapshot size of N = 10, 000 nodes and with 
different acceleration and probability parameters (a,p) 
chosen to generate networks over a range of average con- 
nectivities 0.01 < (k) < 1.0. To do this, we choose dif- 
ferent acceleration parameters in the set a G {0, 1,3,5} 
with probability parameters set according to Eq. The 
resulting slopes of the power law connectivity distribu- 
tions fc -7 averaged over the corresponding acceleration 
parameters varies over the range 1.9 < 7 < 3.0. This 
regime was forbidden prior to taking account of the expo- 
nential cutoffs imposed by the finite ranges. (The choice 
(a,p) = (1, 10~ 5 ) generating 7 ss 2 is roughly equivalent 
to the accelerating simulations of prokaryote gene regu- 
latory networks |2Q|.) Additional network growth would 
see these scale-free distributions undergoing a full tran- 
sition to exponential statistics. 
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FIG. 4: The proportional connectivity distribution l c (k, N) 
for connected nodes under preferential attachment for var- 
ious values of the average network connectivity (k) at a 
snapshot network size of N = 10, 000. The curves nat- 
urally group together according to average network con- 
nectivity, and are characterized by a power law fc -7 with 
slope 7 as shown. Further network growth would eventu- 
ally convert these scale-free distributions to exponential dis- 
tributions. For average connectivity (k) = 0.01, we show 
curves for (a,p) = (0, 0.005), (1, 10" 6 ), (3, 2 x 10" 14 ), (5, 3 x 
10" 22 ),- for (k) = 0.1, (a,p) = (0,0.05),(l,10- 5 ),(3,2 x 
10 _13 ),(5,3 x 10" 21 ); for (k) = 0.4, (a,p) = (0,0.2), (1,4 x 
10" 5 ),(3,8 x 10- 13 ),(5,10- 20 ); for (k) = 1.0, (a,p) = 
(0,0.5),(1,10" 4 ),(3,2 x 10~ 12 ),(5,3 x 10~ 20 ). 



Finally, we briefly consider the size of the largest con- 
nected island in the artificial case of purely preferentially 
attached networks where initially unconnected nodes re- 
main always unconnected. In this particular case, pref- 
erential attachment ensures that every connected node 
belongs to the same interconnected largest island whose 
size then equates to the number of connected nodes. As 
a node is connected only if it gains some connections on 
entry to the network with probability 1 — P(0, k) from 
Eq. QJ the size s of the largest island is 



N 

E 

k=l 



[l-il-ph"- 1 )"] 



(22) 



For small p and small network sizes N, this equates to 
the number of links s « Y2k=i P^ a ~ ^ as eacn connected 
node likely has only one link in sparsely connected net- 
works. In this regime, the largest island grows at an ac- 
celeration rate determined by the acceleration parameter 
a. However, outside this regime, the growth of the largest 
island eventually saturates when almost every new node 
is connected (i.e. when 1 — (1 — pk a ~ v ) k « 1) meaning 
the largest island grows proportionally to network size 
N . This occurs in the limit pk a ~ x — > 1 or N large giving 
s » N. Should a network absolutely need to maintain an 
accelerated growth in the size of its largest island, as con- 
jectured for the regulatory gene network of prokaryotes 
due to competitive pressures, then the point at which 
the largest island makes a transition from accelerated to 



line ar g rowth will represent an upper network size limit 

MM. 



IV. RANDOMLY ATTACHED ACCELERATING 
NETWORKS 

We now turn to consider accelerating and hyper- 
accelerating networks (a > 0) where new links are ran- 
domly distributed over all existing nodes. The final gen- 
erated link distribution will depend on the balance of 
two countervailing trends. The first sees newer nodes 
attracting more links due to acceleration which confers 
an increasing average number of links on younger nodes 
rife, while in contrast, older nodes have a longer time to 
acquire links from the newer nodes. 

Under random attachment, the rate of growth of the 
link number for node rij when the network has grown 
to size k is given under the continuous approximation 
"Elm as 



dljk hk , c 



(23) 



where Ikk = pk a . Here, the rate of growth in the connec- 
tivity Ijk is determined by the number of new links added 
with node rik (Ikk) which are equally proportioned over 
the k existing nodes. The resulting connectivity statistics 
can readily be obtained through integrating this equation 
with the initial conditions Ijj « pj a at time j to obtain 



l jN - 



l+ln4 

j 



a = 0, 



(24) 



Z[N a + (a-l)j a ], a>0. 



These results satisfy Eq. [23 while integration of the link 
numbers over all node numbers j gives the required total 
number of links as in Eq. (As established in Appendix 
lAl these random attachment equations are exactly repro- 
duced by applying a Taylor series expansion of the gen- 
eral equations Eq. ^l a bout the point x — pN a / (3 w 
with the retention only of terms linear in i.) 

In passing, we present the exact distribution for links 
under random attachment. First, note that node fife 
initially receives an average of (j) = pk a links where 
j £ [0, k] with each link formed with probability pk 01 ^ 1 , 
and subsequently receives an average of (jk) — pk a /k = 
pk a ~ l links from itself where jk £ [0, k] with each link 
formed with probability pk a ~ l . The arrival of node rik+i 
is accompanied by an additional p(k + 1) Q new links of 
which an average (jk+i) = p(k+l) a / (k+1) = p(k+l) a ^ 1 
attach to node nj, where jk+i G [0, k + 1] with each link 
formed with probability p(k + l) a ~ 2 , and so on until it 
receives an average of (]n) = pN a /N = pN a ~ 1 links 
from node njf with £ [0, N] with each link formed 
with probability pN a ~ 2 . The joint probability that node 
nk obtains j, jk, ■ ■ ■ , Jjy connections is then 



Pk(j,jk,jk+i, ■ ■ •, jjv) 



(25) 
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The average number of inbound links for node is then 



{] + ]k + ■■■+ Jn) =pk a + 



N 
n—k 



(26) 



which integrates to give Eq. [2] as found by the con- 
tinuum approach. Unfortunately, further work with 
this more exact multivariate distribution is generally in- 
tractable. 

The resulting connectivity IjN verses node age j distri- 
butions are shown in Fig. El along with example simula- 
tion runs at different acceleration parameters a. This fig- 
ure makes clear that the connectivity distribution IjN is 
monotonically decreasing with j for < a < 1 so younger 
nodes are more heavily connected than older nodes on av- 
erage, and monotonically increasing for a > 1 so younger 
nodes are less heavily connected than older nodes on av- 
erage. It is only on the boundary between these two 
regions at a = 1 that the connectivity distribution is flat 
so average node connectivity is independent of node age. 

As previously, the IjN distribution contains informa- 
tion about both node connectivity and node age and so 
approximates network statistics (simulated or observed) 
when all of this information is available. We now calcu- 
late the age independent final link distribution C(k,N) 
as in Eq. 1131 In the case a = 1 we have j ^ j(k), mean- 
ing the connectivity k is independent of the node age j, 
so the delta-function of Eq. ^5] again integrates to give 
S(k — pN). For other cases, we have 



a = 0, 



l_l N a_ak 
1 — a V p 



1/a 



(27) 



, 0<a,u/l. 



The age constraint 1 < j < N translates into the con- 
nectivity constraints k 6 [p,p(l + IniV)] for a = 0, 
k e [ P N a ,pN a /a] for < a < 1, k = pN for a = 1, 
and k £ [pN a /a,pN a ] for a > 1. Via Eq. U21 these rela- 
tions determine the predicted age independent final link 
distribution C(k 7 N) under random attachment to be 



C(k,N) 



(28) 
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1 ( ]\i a — ° 

pJV(l-a) 1 /" \ ly p J 

5{k-pN), 

i i 



a = 0, 

< a < 1, 

a = 1, 
a > 1. 
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FIG. 5: (a) The average connectivity distribution IjN under 
random attachment as a function of node age j for various 
values of the acceleration parameter a. Connectivity is mono- 
tonically decreasing for < a < 1, flat for a = 1, and mono- 
tonically increasing for a > 1. (b) Example simulated net- 
works for various acceleration parameters a with nodes listed 
sequentially left to right from n\ to un with the connectivity 
of each node indicated by vertical lines. The horizontal dashed 
line indicates an average connectivity of two. 



Each of these separate distributions is normalized over 

rk 

k E [fco,fcoo) via J fco °° C(k,N) = 1 where k £ [p, oo) for 
a = 0, k e [ P N a ,pN a /a] for < a < 1, k = pN for 
a = 1, and k £ [pN a /a,pN a ] for a > 1. The average 

connectivity per node (k) = J fco °° kC(k,N) = C/N as re- 
quired over these same ranges. As shown in Fig. [BJ the 
predicted age independent continuous connectivity distri- 
bution C(fc, TV) is monotonically decreasing as a function 
of k for both a < 1 and a > 1, and a delta- function for 
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FIG. 6: The age-independent connectivity distribution 
C(k, N) as a function of acceleration parameter a for ran- 
domly connected networks. The distributions are monoton- 
ically decreasing for a 7^ 1 over the respective ranges k G 
[p, 00) for a = 0, k G [pN a ,pN a /ce] for < a < 1, and 
fc G [piV Q /a,_piV Q ] /or a > 1. 
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FIG. 7: T/ie age-independent discrete connectivity distribu- 
tion Ck as a function of acceleration parameter a for random 
attachment. It is evident that while the unobservable contin- 
uous distributions C(k,N) are distinguishable, the observable 
discrete distributions are virtually indistinguishable. We also 
show the actual points for different network simulations of 
N = 10, 000 nodes with average connectivity (k) = 0.4 with 
a — (triangles), a — 1 (diamonds), a — 3 (circles), and 
a = 5 (squares). 

The continuous connectivity distributions C(k, N) are 
again typically defined over very limited ranges k G 
[fcoi^oo] for acceleration parameters greater than zero 
so many nodes possess fractional connectivity less than 
unity. This again requires that the continuous distri- 
bution C(k,N) be converted into an equivalent discrete 
distribution Ck following the methods of Appendix [5] 
Fig. [7| shows the results of taking full account of the fi- 
nite ranges of the continuous distributions C(k, N) which 
impose exponential cutoffs on the generated discrete dis- 



tribution Ck- It is clear that the exponential cutoff's 
render the observable discrete distributions for differ- 
ent acceleration parameters essentially indistinguishable. 
This result is borne out by simulations of networks with 
N = 10, 000 nodes with average connectivity (k) = 0.4 
for different acceleration parameters (also shown in Fig. 

Again, we illustrate the numerical algorithm of Ap- 
pendix [5] via specific examples for randomly attached 
networks. Consider continuous distributions C(k,N) 
where the boundary point b lies within the range of sup- 
port for the distribution for all acceleration parameters 
except a = 1. That is, we assume pN a < b < pN a /a for 
< a < 1 and pN a /a < b < pN a for a > 1, while for 
a = 1 we assume pN < b. These illustrative choices then 
give 



2 sinh 



C+ = 



r(*-i) 



■k/p) 



r (fe+J) 



0. 



r(fc+§)-l 



r(fc- 



a = 0, 

< a < 1, 
a = 1, 

a > 1. 

(29) 

In this example, the allowed ranges for the connectivity 
k are k e [2, 00) for a = 0, k G [2,pN a /a] for < a < 1, 
and k G [2,pN a ] for a > 1. Our example distributions 
have nonzero support to the left of the boundary point 
b and so can be partitioned to give NC~ nodes labelled 
< j < (NC~ — 1) each with average connectivity kj = 
PjN equal to 



a-l 



f p[l-ln(l-i)], 
-(!-«)(! 



a 



0. 



pN° 



kj = < 



JV / 



pN, 



< a < 1, 
a = 1, 
a > 1. 



(30) 

These values then feed directly into Eq. IB 111 to give the 
total discrete distribution Ck ■ To illustrate this last stage 
of the calculation, consider the case a = 1 where we have 
kj = pjN = pN and pj = p for all nodes < j < N — lsa 
N, and noting the absence of any contribution from the 
right of b so C + — 0, we have 



Ck = C u = 



p k (l-p) 



N-k 



a = 1, 



(31) 



for k G [0, Af]. 

We now turn to establish how the formation of a single 
giant connected component depends on the acceleration 
parameter a. The number of nodes in the largest island 
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FIG. 8: The simulated and predicted size of the largest is- 
land s expressed as a proportion of growing network size for 
various acceleration parameters a. Each network has average 
connectivity (k) — 0.4 at a critical network size N c = 1000 
nodes, and in every case, we assume that the average size of 
smaller islands is s = 7.5. 



s will be a function of network size and will grow as new 
nodes and new connections are added to the network. 
The largest island will grow by one when the new node 
nk forms one link with probability P(l, k) which attaches 
to the largest island with probability s/k, and will grow 
by the average size of all smaller islands s when node 
forms two links with probability P(2, k) either of which 
connects to the largest island with probability s/k while 
the remaining link connects to an external island with 
probability (k — s)/k. Altogether, the rate of growth of 
the largest island goes as 



ds 
dk 



s 
k 

v k a - x 



P(l,k) + 2s 
- p's 



(k 



k 2 



-P(2,k) 



2-£,2a-2 



(fc — s)s, 



(32) 



using Eq. 0]and noting the restriction pk a ~ Y < 1. Nu- 
merical simulations (and some analytically tractable so- 
lutions) indicate that the first term here is negligible com- 
pared to the second, and hereinafter we ignore this first 
term. For initial conditions, we have s(n c ) = 2 at node 
n c where the first link likely appears, see Appendix [5] 
Fig. shows the predicted growth of the largest island 
compared to simulated networks for different acceleration 
parmeters a, where all networks possess an average con- 
nectivity per node of (I) = 0.4 at a critical network size 
of N — 1000, and we assume an average size of smaller 
islands of s = 7.5. There is an evident close connec- 
tion between theoretically predicted curves and observed 
statistics. 



portant biological networks such as prokaryote gene reg- 
ulatory networks are both hyper-accelerating and size 
limited due to network structural transitions. We exam- 
ined accelerating growing networks of nodes connected by 
undirected links which were probabilistically added with 
each new node and either preferentially or randomly at- 
tached to existing nodes. The addition of a probabilistic 
envelope allowing the number of new links added with 
each new node to be an integer greater than or equal 
to zero, allowed us to extend network theory to model 
sparsely connected networks where the majority of nodes 
are entirely unconnected (e.g. prokaryote gene regulatory 
networks). Consequently, the probabilistic envelope also 
allowed us to lift existing constraints on the modelling of 
accelerating networks allowing us to treat decelerating, 
accelerating, and hyper-accelerating networks. These lat- 
ter two classes of networks were shown to be subject to 
transitions in which either a single giant connected com- 
ponent forms or the network condenses into a fully con- 
nected state with exponential statistics. We were able 
to roughly locate these transitions as a function of the 
acceleration and probability parameters and the network 
size. Mean field approximations were compared to net- 
work simulations over a wide range of parameters and 
shown to be consistent. 

The present paper uses mean field network theory to 
model rapidly accelerating networks consisting of many 
unconnected nodes allowing the examination of the sta- 
tistical transitions generated under accelerated growth. 
As such, these extended models will be useful in treat- 
ing models of, for instance, accelerating biological regu- 
latory networks mainly consisting of unconnected nodes 
and displaying network transitions which limit size and 
complexity. Such models are required to explain the ob- 
served evolutionary record of prokaryote gene regulatory 
networks. 



APPENDIX A: RELATIVE WEIGHTS OF 
PREFERENTIAL AND RANDOM 
ATTACHMENT 

In analyzing Eq. 1111 it is first necessary to determine 
appropriate limits on (3 giving access to regimes where 
preferential attachment dominates random attachment 
for accelerating networks. 

Appropriate limits on (3 can be obtained by perform- 
ing a Taylor expansion of Eq. 1111 about the point 
x = pN a I ' (3 ~ while retaining only terms linear in 
x. Using a = in Eq. 1111 we have 



CONCLUSION 



7 - P 
IjN — — 

X 



<1 (f ) - J 



(Al) 



We have examined the network structural transitions 
displayed by accelerating and hyper-accelerating proba- 
bilistic networks motivated by the observation that im- 



Noting d/dx[(l + x)a x ] = a x [l + (1 + x) In a], the leading 
terms of a Taylor expansion immediately reproduce the 
a = result for the random attachment model of Eq. [2] 
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Similarly, using a > in Eq. ^] gives 



1 + x 4t 



- 1 



(A2) 

A straightforward differentiation then gives a Taylor ex- 
pansion whose leading terms exactly equal the a > re- 
sult for the random attachment model of Eq. |^ Hence, 
random attachment entirely dominates when pN a / (3 -C 1 
or equivalently, when 



dom 



pN a 



a)(k). 



(A3) 



For values of (3 < /Random ! preferential attachment will 
influence the final distribution. For sufficiently small 
(3, preferential attachment will dominate (rather than 
merely contribute) and this latter boundary can be lo- 
cated as follows. The first connected node, denoted n c 
(i.e. the c th node), likely appears when the cumulative 
average number of added initial links sums roughly to 
unity, 



fe=i 



pc 



a + 1 



1 



(A4) 



Now the floating end of the new link attached to node 
n c can be either randomly attached to one of the nodes 
ni to n c with probability proportional to c/3, or preferen- 
tially attached to node n c with probability proportional 
to 1 + (3 « 1 in the preferential attachment regime. For 
preferential attachment to dominate, we require c/3 <C 1, 
or equivalently, 



< /9pref 



1 


p 


a + l 


\ 1 


c 


a + 1 




2N?_ 



(A5) 



For the typical hyper-accelerating networks considered 
here, this last constraint ensures that preferential at- 
tachment dominates only when (3 is so small as to be 
essentially zero, and in this paper, we analyze preferen- 
tial attachment using the setting (3 = 0. 



APPENDIX B: EXPONENTIAL CUTOFFS IN 
HYPER-ACCELERATING NETWORKS 

An arbitrary continuous network connectivity distribu- 
tion C(k, N) defined over a finite range [kg, fcoo] is essen- 
tially unobservable, and in actuality, generates an equiva- 
lent discrete connectivity distribution Ck with k e [0, N] 
which can be compared to observed statistics. Here, we 
show how to calculate the observable discrete distribu- 
tion from the theoretically predicted but unobservable 
continuous distribution. 

The first step is to partition the continuous distribution 
C{k, N) into two parts at an arbitrarily chosen boundary 
point b of order unity. We choose b = 3/2 as shown in 




FIG. 9: A continuous distribution C(k,N) defined over the 
finite range [ko,koo] generates an observable discrete distri- 
bution Ch for fc = 0, 1,2, .... The C(k,N) distribution lying 
to the left of the boundary b = 3/2 represents network nodes 
with fractional average connectivity distributions, and is parti- 
tioned into regions each of area 1/N with boundaries [kj, fej+i) 
for j > 0. The C(k,N) distribution lying to the right of the 
boundary b — 3/2 represents network nodes with integral av- 
erage connectivity, and is partitioned into regions as shown. 



Fig. |5J Then, that proportion of the continuous distribu- 
tion C(k, N) lying to the right of b can be considered to 
contribute to the long tail of the observed discrete distri- 
bution Cfc in the normal way. That is, the proportion of 
the continuous distribution in the region [k — 1/2, fc+1/2) 
for integral k equal to 



fe+l/2 



C(k,N) dk, 



(Bl) 



fc-l/2 



can be entirely assigned to the discrete distribution bin 
Cfc. The total proportion of the continuous distribution 
C(k, N) so assigned is 



k=b+l/2 



C(k,N) dk, 



(B2) 



and this proportion of the distribution describes the con- 
nectivity of a total of NC + nodes. When this proportion 
is close to unity as usually applies for long tailed distribu- 
tions, nothing further need be done and the distribution 
bins — Ck then equate to the predicted discrete ob- 
servable distribution. 

However, when C + is significantly less than unity as 
applies for hyper-accelerating networks, it is necessary 
to assign the remaining proportion of the continuous dis- 
tribution lying to the left of the chosen value b to the 
discrete distribution bins Ck ■ The proportion of the con- 
tinuous distribution remaining to be assigned is 



C~ = 



C(k,N) dk = 



(B3) 



with this proportion of the distribution describing the 
connectivity of a total of NC~ nodes. The best way 
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to understand how this remaining assignment is done is 
through a successive partitioning of the usual normaliza- 
tion constraint 



C(k,N) dk 

ko 

C(k,N) dk+ C(k,N) dk 

ko Jb 



E 

3=0 



•i+i 



C(k,N) dk + C+. (B4) 



Here, the continuous distribution to the left of the point 
k = b is partitioned into NC~ regions with respective 
boundaries [kj, fej+i) for kj G [ko,b] and kj < fcj+i while 
as noted above, that to the right of k = b is partitioned 
into potentially an infinite number of regions with bound- 
aries [k — 1/2, k + 1/2) for integral k > b. The second 
contribution to the discrete bin Ck comes from the NC~ 
regions of the continuous distribution lying to the left of 
b. We would like each of these NC~ regions to corre- 
spond to a single node of the network with non-integral 
average connectivity, and achieve this by choosing the 
regional boundaries kj such that 



C(k,N)dk=±, je[o,NC--i]. 



(B5) 



Here, each region defines an area of exactly 1/N so j 
regions have a combined area of j/N, see Fig. [5J The 
average connectivity of the node corresponding to the j 
region is approximately 



(kj) — kj 



(B6) 



and is typically non-integral. (Other alternatives such 
as (kj) = (kj + fcj_i)/2 might be chosen but the differ- 
ence is negligible for large N.) As any given node can, in 
practise, only possess an integral number of connections, 
these non-integral average connectivities require that the 
link numbers for these nodes be assigned probabilistically 
over the range k G [0, N] so as to give the required av- 
erage. This is achieved by assigning an integral number 
lj of links to this region with lj G [0, TV] according to the 
probability distribution 



P r (lj 



with probabilities pj chosen to satisfy 



Pj 



N 



(B8) 



to ensure that this distribution has average (lj) — PjN — 
(kj). We note that later results will not be overly depen- 
dent on whether the P r distribution chosen here is bino- 
mial in form, or some other reasonable distribution. Sum- 
ming over all the NC~ regions to the left of b, the pro- 
portion of nodes possessing an integral number of lj = k 
links is then 



Cl 



1 

N 



(NC--1) 

E 

3=0 



N 



P](l-Pj) 



N-k 



(B9) 



for k G [0, N] and zero otherwise. This is normalized 



according to Ylk=o^k = @~ . Consequently, the total 
proportion of the network possessing a discrete number 
of k links is expected to be 



C fc - C k + Ct. 



(BIO) 



As required, this distribution is normalized and has aver- 
age (k) = L/N. This gives the discrete probability that 
a node has k links for k G [0, oc). It is also common 
to calculate the related probability that connected nodes 
have k links and this is given by C' k = Cfc/(1 — Co) for 
k > 1. 

In summary, for the general case of arbitrary contin- 
uous distributions C(k,N) defined over the range k G 
[fcojfcoo], the predicted discrete distribution is 



Ck — 



Ck+ct 

NC~ 



N 



5 £ (*)#'-»>"-' 



3=0 

rk+1/2 
+ / C(k,N) dk, 

Jk-X/2 



(Bll) 



where C = f k C(k,N) dk, pj = kj/N, kj is chosen to 
satisfy j/N = j^ 3 C(k,N) dk, and the partition point b 



(B7) is arbitrarily chosen to be close to unity. 
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